Multi-Frame Feature Aggregation for Real-Time Instrument Segmentation in Endoscopic Video

Deep learning-based methods have achieved promising results on surgical instrument segmentation. However, the high computation cost may limit application of deep models to time-sensitive tasks such as online video analysis for robotic-assisted surgery. Moreover, current still suffer from challenging conditions in images various lighting and presence blood. We propose a novel Multi-frame Feature Aggregation (MFFA) module aggregate frame features temporally spatially recurrent mode. By distributing load feature extraction over sequential frames, we can use lightweight encoder reduce costs at each time step. public videos usually are not labeled by frame, so develop method that randomly synthesize sequence single assist network training. demonstrate our approach achieves superior performance corresponding deeper segmentation two surgery datasets.